Bayesian controller fusion: Leveraging control priors in deep reinforcement learning for robotics

We present Bayesian Controller Fusion (BCF): a hybrid control strategy that combines the strengths of traditional hand-crafted controllers and model-free deep reinforcement learning (RL). BCF thrives in robotics domain, where reliable but suboptimal priors exist for many tasks, RL from scratch remains unsafe data-inefficient. By fusing uncertainty-aware distributional outputs each system, arbitrates between them, exploiting their respective strengths. study on two real-world tasks involving navigation vast long-horizon environment, complex reaching task involves manipulability maximisation. For both these domains, simple handcrafted can solve at hand risk-averse manner do not necessarily exhibit optimal solution given limitations analytical modelling, controller miscalibration variation. As exploration is naturally guided by prior early stages training, accelerates learning, while substantially improving beyond performance prior, as policy gains more experience. More importantly, risk-aversity ensures safe deployment, dominates action distribution states unknown to policy. additionally show BCF’s applicability zero-shot sim-to-real setting its ability deal with out-of-distribution real world. promising approach towards combining complementary robotic control, surpassing what either achieve independently. The code supplementary video material are made publicly available https://krishanrana.github.io/bcf .